Auxiliary Lemmas : Proof of Lemma 3

Supplementary Material 6 Auxiliary Lemmas: Proof of Lemma 3 Proof. We can rewrite (6) as an optimization problem over the ℓ 1 /ℓ 2 ball of radius C for some C(λ n) < ∞. 1,2 = C for all optimal primal solution Θ \r. By definition of the ℓ 1 /ℓ 2 subdifferential, we know that for any column u ∈ V \{r}, we have (ˆ Z \r) u 2 ≤ 1. Considering the necessary optimality condition ∇ℓ (ˆ Θ \r) + λ n ˆ Z \r = 0, by complementary slackness condition, we have ⟨ Θ \r , ˆ Z \r ⟩ − C = ⟨ Θ T \r , ˆ Z \r ⟩ − Θ \r 1,2 = 0. Now if for an arbitrary column u ∈ V \{r}, we have (ˆ Z \r) u 2 < 1 and (Θ \r) u ̸ = 0 then this would contradict the condition that ⟨ Θ \r , ˆ Z \r ⟩ = Θ \r 1,2. For this restricted problem, if the Hessian sub-matrix is positive definite, then the problem is strictly convex and it has a unique solution. In this section, we point out the key properties of the gradient, Hessian and derivative of the Hessian for the log-liklihood function. These properties are used to prove the concentration lemmas.